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A  COGNITIVE  MODEL  OF  PEDAGOGICAL  QUESTION  ASKING 


EXECUTIVE  SUMMARY 


Requirement : 

The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
(ARI)  conducts  research  in  support  of  new  technologies  for  Army  training.  One 
area  of  research  is  that  of  developing  computer  training  systems  with  soldier- 
machine  tutorial  interfaces  with  natural  language  capability.  This  report 
describing  a  theory  of  pedagogical  question  asking  provides  a  cognitive  basis 
for  future  research  in  this  area. 


Procedure : 

The  author  of  this  report  describes  a  cognitive  theory  of  pedagogical 
question  asking  in  a  learning-by-doing  paradigm  based  on  a  spreading  activa¬ 
tion  memory  framework.  This  theory  is  part  of  an  in-house  research  effort 
investigating  the  natural  language  requirements  in  tutorial  dialogues  for 
military  training  systems. 


Findings : 

Pedagogical  questions  function  according  to  specific  conversational  rules 
in  order  to  obtain  instructional  information  for  the  learner.  Questions  func¬ 
tion  as  probes  in  memory  to  focus  activation  of  the  knowledge  being  processed 
at  the  time  a  question  is  posed.  Further  functions  for  these  questions 
include  articulating  discrimination  and  generalization  learning  of  the 
knowledge . 


Utilization  of  Findings: 

The  purpose  of  this  report  is  to  describe  a  theoretical  model  of  peda¬ 
gogical  question  asking.  Subsequent  planned  research  will  assess  the  useful¬ 
ness  of  this  model  for  describing  soldier  trainees'  knowledge  states  during 
computer-based  learning.  A  target  domain,  the  97E  military  interrogator,  will 
explore  the  usefulness  of  this  model  for  training  question  asking  skills  in 
foreign  languages. 
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A  COGNITIVE  MODEL  OF  PEDAGOGICAL  QUESTION  ASKING 


INTRODUCTION 

Question  asking  is  an  integral  part  of  instruction.  Military  students 
will  often  ask  questions  to  help  them  comprehend  what  they  are  learning.  As 
an  instructional  strategy,  question  asking  has  received  attention  in  the  re¬ 
search  community  (Collins,  Brown  &  Newman,  1987;  Palincsar  &  Brown,  1984),  but 
very  little  work  has  been  done  examining  the  cognitive  aspects  of  student 
question  asking.  The  study  of  question  asking  may  have  important  consequences 
for  understanding  the  kinds  of  knowledge  a  student  needs  and  when  during  skill 
acquisition.  Moreover,  pedagogical  questions  may  reflect  how  students  are 
reasoning  about  (Norman,  1982)  and  organizing  the  knowledge. 

The  use  of  questioning  in  a  tutorial  dialogue  dates  back  to  Socrates  who 
believed  that  the  appropriate  question,  when  posed  to  the  student,  would  cause 
the  student  to  "remember"  the  correct  response.  He  believed  that  we  possess 
all  the  knowledge  of  the  world  in  our  minds,  but  don't  always  have  access  to 
this  knowledge.  "If  someone  will  keep  asking  the  student  these  same  questions 
often  and  in  various  forms,  you  can  be  sure  that  in  the  end  he  will  know  about 
them  as  accurately  as  anybody"  (Plato,  1924).  Following  the  Socratic  method, 
question  asking  by  the  tutor  serves  to  refine  and  modify  the  student's  knowl¬ 
edge  structure  by  referring  to  the  underlying  causal  model  for  the  domain. 
Tutor -generated  questions  are  generally  intended  to  evaluate  what  the  student 
knows  about  the  domain  by  requesting  that  the  student  provide  a  response. 
Through  the  mechanism  of  tutorial  question  asking,  responding,  and  tutorial 
explanation,  the  student  will  eventually  acquire  what  it  is  he  or  she  needs  to 
know  from  the  instruction  presented. 

However,  tutorial  dialogues  which  enable  reciprocal  communication  also 
include  student-generated  questions.  Question  asking  by  students  can  serve  as 
a  self-directed  monitor  during  instruction  to  guide  acquisition  of  the  knowl¬ 
edge  and  to  request  instructional  information  that  is  necessary  for  the  cor¬ 
rect  encoding,  organization,  and  comprehension  of  the  knowledge.  The  work 
addressed  here  will  focus  on  the  function  of  student  questioning  in  tutorial 
dialogues  and  how  the  questions  serve  to  monitor  the  student's  cognitive 
processes  and  reflect  their  reasoning  about  the  knowledge  during  skill  ac¬ 
quisition.  The  domain  used  to  discuss  the  question  model  is  Lotus  1-2-3,  a 
procedural  computer  command  language;  the  instructional  context  is  a  tutorial 
dialogue  to  guide  computer-based  learning  by  doing. 

Pedagogical  question  asking  is  based  upon  the  following  premises.  First, 
there  is  a  unique  question  system  that  functions  in  a  specialized  manner  in 
pedagogy.  Secondly,  within  this  system,  student-generated  questions  function 
to  aid  the  student  in  understanding  new  knowledge  (Brown  &  DeLoache,  1978; 
Bransford,  1979;  Collins,  1977).  And  finally,  the  ability  of  a  student  to 
articulate  questions  during  learning  is  based  on  the  availability  and  knowl¬ 
edge  appropriate  structures  in  memory  (Miyake  &  Norman,  1979).  Furthermore, 
evidence  exists  (Chi  &  Glaser,  1980;  Swartz,  1987)  that  novices  do  in  fact  ask 
many  questions  when  learning  a  new  skill,  even  if  knowledge  appropriate  struc¬ 
tures  are  not  yet  formed. 


The  cognitive  processes  underlying  the  proposed  model  of  pedagogical 
question  asking  are  based  on  the  theory  of  spreading  activation.  To  begin,  I 
will  argue  that  pedagogical  questions  reflect  in  part  self-direc ted  probes  in 
memory  which  function  as  a  control  mechanism  for  focusing  activation  at  a 
particular  knowledge  node  and  maintaining  this  information  in  working  memory 
during  processing.  In  articulating  a  question  the  student  has  searched  mem¬ 
ory  for  the  requisite  information,  found  it  was  not  available,  and  evaluated 
a  requirement  for  asking  a  question  to  obtain  what  is  needed  for  understand¬ 
ing  or  to  perform  some  activity  in  the  task  environment.  Secondly,  I  claim 
that  pedagogical  questions  function  as  organizers  for  the  knowledge  through 
discrimination  and  generalization  processes  as  students  evaluate  what  they 
know  with  the  feedback  received  from  tutorial  responses  and  task  goal  end 
s  ta  tes . 

The  kinds  of  questions  asked  in  a  particular  domain  or  military  occupa¬ 
tional  specialty  (MOS)  should  provide  information  that  is  descriptive  of  a 
student's  current  knowledge.  Question  analysis  can  be  used  for  defining  the 
MOS-specific  goals,  operators,  and  methods  a  student  is  using  or  needs  at  a 
given  time  during  the  instructional  sequence.  Mapping  the  questions  asked  at 
a  particular  point  during  the  acquisition  process  onto  the  conceptual  repre¬ 
sentation  for  the  domain  should  indicate  what  kind  of  knowledge  is  being 
processed  by  a  student.  Using  a  spreading  activation  mechanism  as  the  frame¬ 
work  for  accessing  the  developing  domain  representation,  we  should  be  able  to 
take  these  mapped  questions  and  derive  knowledge  state  activation  patterns 
for  a  given  point  in  time.  As  new  material  becomes  instantiated  and  organ¬ 
ized  in  the  student's  internal  representation,  these  patterns  should  shift 
from  one  area  of  the  network  to  another.  This  knowledge  state  information 
may  have  potential  for  facilitating  the  development  of  tutorial  dialogues  and 
ultimately  instructional  delivery  in  computer-based  learning  by  providing  a 
more  clear  understanding  of  what  information  the  student  is  requesting  and 
processing  at  the  time  a  question  is  asked. 

This  paper  describes  a  cognitive  model  of  the  processes  that  underlie 
student  pedagogical  questions.  Subsequent  reports  will  provide  empirical 
data  in  support  of  the  model  presented  here.  I  will  begin  with  a  descrip¬ 
tion  of  the  structure  of  the  pedagogical  question  system.  The  next  section 
will  outline  the  theoretical  basis  of  question  asking.  I  will  present  a 
cognitive  model  of  question  asking  and  illustrate  how  questions  are  used  by 
students  for  metacognitve  control  to  focus  activation  in  the  developing  rep¬ 
resentation  and  how  these  same  questions  reflect  a  discrimination  and  gener¬ 
alization  learning  function.  The  last  section  will  discuss  how  this  model 
can  be  used  for  deriving  knowledge  state  activation  patterns  and  the  poten¬ 
tial  role  this  may  have  for  student  modeling  in  the  development  of  computer- 
based  tutorial  dialogue  interfaces  in  military  training  systems. 

THE  PEDAGOGICAL  QUESTION  SYSTEM 

Specialized  language  subsystems  exist  within  human  communication  which 
enable  us  to  exchange  information  in  a  particular  manner.  For  example:  for¬ 
mal  debate,  rhetoric,  and  question  asking.  Each  subsystem  conanunica tes  in¬ 
formation  in  specialized  ways  for  the  particular  purpose  of  the  system.  The 
formal  debate  subsystem  uses  communication  rules  which  foster  organized  argu¬ 
mentation  between  individuals  to  defend  or  attack  a  given  proposition. 


Rhetoric  employs  language  to  persuasively  influence  the  thoughts  and  actions 
of  others.  Question  asking  functions  as  an  expression  of  inquiry  that  in¬ 
vites  a  response  or  casts  doubt  on  some  piece  of  information.  The  question 
subsystem,  one  of  the  many  specialized  subsystems  in  human  communication,  is 
subject  to  its  own  unique  set  of  linguistic,  contextual,  and  usage  rules  to 
achieve  its  communicative  purpose. 

The  General  Question  System 

Searle  (1969)  defines  a  question  as  a  special  case  of  request.  He  dis¬ 
tinguishes  between  'real'  questions  that  request  information  and  'test'  ques¬ 
tions  that  request  that  the  hearer  provide  information.  Belnap  and  Steel 
(1976)  believe  that  a  question  makes  an  implicit  request  regarding  an  answer 
the  respondent  is  to  provide.  Hairah  (1973)  refers  to  'standard'  questions 
in  pedagogy  in  which  the  questioner  knows  what  the  problem  is  and  how  to 
articulate  a  question  in  order  to  receive  an  appropriate  response.  Questions 
can  also  be  speech  acts  that  cast  doubt,  however,  I  will  not  include  this 
type  of  question  in  the  discussion  of  this  system.  Each  of  these  definitions 
hold  subtle  nuances  in  meaning  and  the  reader  can  appreciate  why  the  develop¬ 
ment  of  a  psychological  model  of  question  asking  remains  a  challenging  task. 
Here  I  use  the  notion  of  a  'real'  question  as  being  uttered  with  a  goal  of 
requesting  information.  Rhetorical  questions,  questions  which  cast  doubt, 
and  questions  whose  motivations  are  governed  by  other  factors  other  than 
obtaining  information  will  not  be  considered.  A  question  can  be  thought  of 
as  a  special  case  of  proposition.  In  question  utterances  a  proposition  is 
expressed,  however,  with  some  of  the  propositional  contents  missing.  The 
interrogative  pronoun  in  this  question  can  be  thought  of  as  an  anaphoric 
referent  (Searle,  1969;  Grice,  1975;  Belnap  &  Steel,  1976)  to  these  missing 
elements. 

As  a  framework  in  which  these  questions  function,  I  propose  five  general 
principles:  1)  to  obtain  information  through  inquiry,  2)  for  each  question 

there  exists  an  appropriate  response,  3)  questions  identify  problems,  4) 
question  form  must  be  appropriate  for  the  domain,  and  5)  the  intent  of  the 
question  must  be  understood.  These  principles  are  discussed  in  the  next 
section,  however,  their  use  to  structure  the  question-answer  exchange  may 
vary  between  domains.  The  following  provide  two  examples  of  how  question 
subsystems  function  differently. 

In  military  interrogation,  the  question  system  plays  a  major  role  in 
obtaining  information  from  a  source  during  the  interrogation  process.  This 
communication  exchange  is  highly  controlled  and  manipulated  by  the  military 
interrogator.  The  interrogator  poses  carefully  constructed  questions  de¬ 
signed  to  elicit  and  verify  specific  responses  from  the  prisoner  (source). 

The  prisoner  is  not  allowed  to  ask  questions  of  the  interrogator.  Often 
prisoners  are  recalcitrant  and  unreliable  respondents.  Therefore,  the  mili¬ 
tary  interrogator  must  decide  upon  a  certain  approach  to  use  in  the  question¬ 
ing  phase  of  the  interrogation.  This  approach  technique  and  the  ways  in 
which  the  question  system  is  manipulated  to  extract  information  from  the 
source  are  unique  to  this  domain.  In  military  interrogation  the  interrogator 
exercises  complete  control  of  the  communication  function  of  the  question 
system,  thus  violating  Grices's  (1975)  cooperative  conversation  principles. 
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In  medical  diagnosis,  the  question  system  operates  quite  differently  with 
a  more  cooperative  exchange  occuring.  The  physician  poses  the  questions  to 
the  patient  with  the  purpose  of  obtaining  symptomatic  information  that  will 
lead  the  physician  to  identifying  the  correct  medical  diagnosis.  The  ques¬ 
tions  asked  are  formulated  to  test  certain  hypotheses  which  are  either  con¬ 
firmed  or  disconfirmed  on  receipt  of  the  patient  response.  The  questioning 
sequence  adapts  to  the  cummulative  effect  of  the  symptomatic  information 
provided  by  the  patient  and  new  questions  generating  new  hypotheses  will  be 
asked  if  medically  warranted.  When  the  physician  has  evaluated  and  tested 
all  plausible  hypotheses  and  feels  certain  of  a  diagnosis,  the  questioning 
process  terminates.  While  the  physician  has  control  over  the  questions 
asked,  his  or  her  motivation  for  questioning  is  different  than  that  of  the 
military  interrogator. 

In  a  pedagogical  domain,  the  question  system  operates  in  yet  another 
manner  than  the  two  examples  cited  above.  The  following  discussion  will 
provide  a  basis  for  understanding  how  a  specialization  in  pedagogy  effects 
the  general  principles  by  which  the  system  operates. 

Specialization  of  the  Question  System  in  Pedagogy 

In  the  pedagogical  question  system,  the  main  purpose  of  student  question 
asking  is  to  obtain  and  process  instructional  information.  In  this  system, 
basic  pedagogical  questions  will  be  similar  to  those  defined  by  Searle,  real 
(student)  and  test  (tutor)  questions.  Who  obtains  the  information  through 
questioning,  the  tutor  or  the  student,  will  affect  the  functional  mechanism 
by  which  questions  operate.  Harrah  (1973)  calls  for  a  theory  of  pedagogical 
questions  that  would  encompass  both  'standard'  questions  (Searle' s  real  ques¬ 
tion)  and  test  questions  (Searle' s  test  question)  in  the  pedagogical  question 
system.  The  test  question  is  that  which  the  tutor  poses  in  order  to  evaluate 
what  the  student  knows  or  how  well  certain  concepts  are  understood.  Tradi¬ 
tionally,  researchers  and  educators  have  concentrated  on  studying  this  kind 
of  questioning  in  pedagogy.  I  will  focus  primarily  on  the  function  of  stan¬ 
dard  or  ' real ' s tudent  questions  in  the  discussion  of  the  question  principles 
and  question  system  in  pedagogy.  In  order  to  understand  how  the  pedagogical 
question  system  functions,  we  must  describe  the  special  conditions  surround¬ 
ing  the  five  question  asking  principles,  certain  operational  features  unique 
for  students  using  the  system,  and  the  predictable  violations  that  occur 
quite  readily  in  pedagogy. 

Principle  1;  Obtaining  Information  Through  Inquiry 

In  the  general  question  system,  the  individual  who  wants  to  get  some  type 
of  information  poses  a  question  intended  to  solicit  some  response.  This  fun¬ 
damental  principle  is  based  on  the  assumption  of  cooperative  communication 
(Grice,  1975).  I  assume  the  same  cooperative  exchange  in  the  pedagogy 
wherein  those  involved  in  the  tutorial  dialogue  (student  and  tutor)  communi¬ 
cate  freely  for  instructional  purposes.  Generally,  there  are  no  obvious 
violations  of  this  principle  in  pedagogy.  Evidence  exists  that  novice  stu¬ 
dents  generally  ask  a  lot  of  questions  (Chi  &  Glaser,  1980)  and  that  some  of 
these  questions  are  used  to  familiarize  the  student  with  the  situation 
(McKoewn,  1982)  before  making  requests  to  obtain  specific  information  related 
to  a  given  task  problem.  Thus  we  can  expect  this  principle  to  apply  often  in 


the  early  stages  of  skill  acquisition  and  throughout  learning  as  instruction¬ 
al  information  is  requested.  Students  will  ask  questions  to  obtain  specific 
kinds  of  information  to  verify  facts,  test  hypotheses,  complete  understanding 
of  specified  details,  instantiate  a  particular  procedure  or  establish  causal 
relationships  as  the  subject  matter  is  learned. 

Several  presuppositions  for  asking  a  pedagogical  question  under  this 
principle  are  that  the  speaker,  a  student,  believes  a  response  exists,  knows 
what  problem  exists,  and  can  articulate  an  interrogative  phrase  in  a  certain 
form  that  will  obtain  the  requisite  information  to  solve  the  problem.  These 
presuppositions  can  be  explained  by  the  remaining  principles  extant  in  the 
pedagogical  question  system 

Principle  2;  For  Each  Question  There  Exists  an  Appropriate  Response 

Questions  operate  under  the  principle  for  obtaining  information  with  an 
implicit  belief  by  the  speaker  that  a  response  exists.  This  second  principle 
is  based  on  the  assumption  that  a  speaker  will  most  likely  ask  a  question 
when  he  or  she  believes  a  response  is  available.  The  topic  of  delivering 
appropriate  responses  to  questions  has  been  the  focus  of  much  research  in 
artificial  intelligence  (McKoewn,  1982;  Mays,  Joshi,  &  Webber,  1985;  Pollack, 
1985).  Mays  et  al  summarize  several  pertinent  issues  in  response  behavior 
which  may  effect  the  validation  of  this  principle.  Responses  may  provide 
more  information  than  is  literally  requested,  justify  or  explain  an  answer, 
correct  misconceptions  reflected  from  the  query,  and  explain  about  the  struc¬ 
ture  and  content  of  the  domain  knowledge.  Thus  appropriateness  of  response 
implies  that  it  address  the  true  intent  of  the  question.  In  pedagogy  the 
tutor's  response  to  student  questions  will  play  an  important  role  in  guiding 
the  student's  learning.  These  response  behavior  issues  raise  many  problems 
in  deciding  on  the  appropriateness  of  an  instructional  response.  How  to  to 
solve  the  problems  mentioned  above  remains  as  yet  unknown  although  many 
researchers  are  actively  addressing  this  topic. 


In  a  pedagogical  setting,  either  the  student  or  the  tutor  will  be  the 
respondent  depending  on  who  asks  the  question.  If  the  respondent  is  believed 
by  the  speaker  to  be  able  to  deliver  an  appropriate  response,  then  the  speaker 
will  be  more  inclined  to  pose  a  question.  In  the  ideal  environment  with  both 
the  student  and  teacher  participating  as  speaker- respondent  in  the  pedagogical 
question  system,  there  is  no  reason  why  an  appropriate  answer  should  not  be 
available  from  the  tutor  as  respondent.  We  assume  that  the  tutor  knows  the 
domain  well  and  also  how  to  answer  a  variety  of  student  questions.  Indeed, 
the  student  as  questioner  implies  his  or  her  trust  in  the  assumption  that  the 
tutor  will  be  capable  of  answering  the  question  with  an  appropriate  instruc¬ 
tional  response.  Thus  in  the  case  of  student  question  asking,  we  don't  ex¬ 
pect  any  violations  of  this  principle.  However,  when  the  tutor  poses  the 
question,  the  student,  who  does  not  possess  the  expert  knowledge  representa¬ 
tion  for  the  domain,  may  have  difficulty  in  providing  an  answer.  Thus,  in 
the  case  of  tutor  question  asking  we  can  expect  common  violations  of  this 
principle  by  the  student.  In  fact,  certain  types  of  these  violations  in 
defined  domains  should  be  predictive  when  memory  processing  constraints  and 
level  of  learning  are  taken  into  account.  Students  may  commit  errors,  forget 
knowledge,  develop  misconceptions  about  what  they  are  learning;  all  of  which 
will  affect  their  responses  to  tutor-generated  questions. 


Principle  3:  Problems  Hust  be  Identified  to  Ask  a  Question 


To  ask  a  question  the  questioner  must  need  the  requested  information.  A 
presupposition  for  asking  a  question  under  this  principle  is  that  the  speaker 
knows  what  problem  exists  and  can  articulate  a  question  that  will  obtain  the 
requisite  information  to  solve  the  problem  (Harrah,  1973).  The  question 
itself  points  to  what  the  student  believes  the  problem  to  be.  Improperly 
identified  problems,  however,  will  affect  the  utility  of  the  question  for 
soliciting  the  appropriate  response  and  guiding  subsequent  comprehension 
processes.  In  the  pedagogical  setting,  the  problem  identified  is  usually  a 
lack  of  adequate  domain  knowledge  or  a  misunderstanding  about  some  piece  of 
instructional  information.  A  basic  assumption  here  is  that  if  no  problem  is 
identified  then  no  question  is  necessary. 

This  principle  is  violated  when  the  student  misidentif ies  the  problem. 
These  difficulties  are  most  prevalent  at  the  beginning  stages  of  learning. 
Although  students  know  that  they  do  not  know  what  they  need  to  know,  they  may 
not  be  able  to  precisely  locate  the  task-appropriate  problem  to  facilitate 
their  understanding.  Thus  their  questions  may  not  be  useful  for  the  problem 
at  hand.  Ue  can  anticipate  that  the  student  will  certainly  be  able  to  iden¬ 
tify  problems  and  thus  the  basic  premise  of  this  principle  will  not  be  vio¬ 
lated.  However,  recognizing  students'  incorrectly  identified  problems  is 
important  for  delivering  appropriate  responses,  tutorial  remediation,  and 
instructing  the  students  in  the  correct  use  of  the  question  system  in 
pedagogy. 


Principle  4:  Question  Form  Must  be  Appropriate  to  the  Domain 

In  a  given  situation,  a  questioner  will  identify  the  need  to  obtain  some 
information  in  order  to  solve  some  problem.  He  or  she  must  then  articulate 
the  question  in  a  form  appropriate  for  obtaining  the  requested  information. 
The  ability  to  articulate  such  a  question  assumes  that  the  questioner  is 
familiar  with  the  domain  or  with  the  situation.  Similarly,  that  enough 
requisite  knowledge  is  available  that  will  permit  the  questioner  to  evaluate 
any  potential  problem  and  subsequently  pose  a  question  if  further  information 
is  required.  In  pedagogy,  this  principle  will  often  be  violated,  especially 
in  the  early  stages  of  learning. 

Novices  have  difficulty  generating  questions  of  the  appropriate  form 
because  they  lack  domain-specific  knowledge  (Miyake  &  Norman,  1979).  Stu¬ 
dents  will  always  be  able  to  identify  problems  even  if  they  involve  certain 
misconceptions  about  the  domain  or  misidentify  the  appropriate  problem.  How¬ 
ever,  the  ability  to  articulate  questions  in  a  form  which  correctly  identi¬ 
fies  what  it  is  the  student  wants  to  know  is  not  always  easy.  Often 
questions  will  be  vague  and  full  of  anaphoric  and  ambiguous  reference  as 
students  attempt  to  articulate  a  question.  Violations  of  this  principle  will 
effect  the  basic  communicative  purpose  of  the  question  system.  Students  who 
are  unable  to  articulate  a  question  of  the  correct  form  will  risk  confusing 
the  intent  of  their  question  and  impede  receiving  the  needed  ins  true tional 
response.  Yet,  in  pedagogy,  the  "ideal"  tutor  is  mindful  of  such  violations 
and  can  compensate  for  ill- formed  questions  by  heuristic  interpretation  of 
the  question,  what  is  perceived  by  the  tutor  as  the  intent  of  the  question, 
and  the  instructional  context.  Indeed,  it  is  under  this  principle  that  much 
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instructional  tutoring  may  occur.  For  example,  in  reciprocal  teaching 
(Palincsar  &  Brown,  1984)  students  are  taught  how  to  articulate  questions  of 
the  correct  form  in  order  to  help  improve  their  reading  comprehension  strate¬ 
gies. 

Principle  5:  Intent  of  the  Question  Must  be  Understood 

In  the  pedagogical  question  system,  this  principle  is  violated  somewhat 
less  frequently  on  the  part  of  the  tutor  as  respondent,  a  domain  expert  with 
all  of  the  requisite  knowledge  available  for  tutoring  the  subject  matter.  As 
subject  matter  expert  and  one  who  presumably  knows  the  domain  knowledge  as 
well  as  student  reactions,  responses,  and  common  misconceptions  during  learn¬ 
ing,  the  tutor  should  for  the  most  part  be  able  to  interpret  the  intent  of 
even  the  most  poorly  articulated  student  question.  Admittedly,  this  places  a 
heavy  burden  on  the  tutor.  Nevertheless,  as  in  general  discourse  processing, 
intent  may  be  misunderstood  and  a  clarification  dialogue  between  tutor  and 
student  may  be  initiated  to  facilitate  identification  of  the  true  intent  of  a 
student  question.  The  intent  of  a  particular  question  requires  that  the 
respondent  appropriately  interpret  the  question.  If  the  intent  is  not  under¬ 
stood  because  the  student  failed  to  articulate  a  question  of  the  appropriate 
form  given  a  certain  problem,  then  principles  three  and  four  above  may  have 
been  violated.  If  the  intent  of  the  question  is  misunderstood  because  the 
tutor  failed  to  correctly  interpret  an  appropriately  articulated  question, 
then  this  principle  will  be  violated. 

Interpretation  of  question  intent  depends  on  several  linguistic  factors: 
semantics,  pragmatics,  and  context.  Without  considering  all  of  these  ele¬ 
ments,  the  intent  of  the  question  may  be  misinterpreted.  Discourse  process¬ 
ing  and  topic  intention  in  topic  research  (Graesser  &  Black,  1984;  Robertson, 
Black  &  Johnson,  1981;  Grimes,  1975)  have  investigated  the  complexities  in¬ 
volved  in  accurately  interpreting  intention  in  discourse.  This  problem  is  a 
difficult  one,  yet  crucial  to  the  respondent's  ability  to  provide  an  appro¬ 
priate  response  to  the  student's  question  and  to  enable  a  cooperative 
exchange . 

Summary 

Several  predictable  violations  of  these  principles  occur  in  the  ques¬ 
tion  system  in  pedagogy.  Students  will  make  errors  identifying  problems, 
develop  misconceptions,  and  pose  poorly  articulated  questions  during 
learning.  The  enlightened  tutor,  aware  of  these  possible  violations,  can 
be  better  prepared  to  engage  in  meaningful  exchanges  with  the  student 
during  instruction.  Recognition  and  understanding  of  these  violations  are 
requisite  to  good  tutoring  and  may  also  foster  instructional  approaches  to 
guide  the  student  toward  better  use  o£  these  communication  systems  during 
learning.  Indeed  several  researchers  are  exploring  this  type  of  approach 
to  learning  (Collins,  Brown,  &  Newman  1987;  Palincsar  &  Brown,  1984). 

With  this  background,  we  can  now  proceed  to  a  discussion  of  the  theoreti¬ 
cal  framework  for  the  psychological  processes  underlying  the  function  of 
this  question  system  during  learning. 


v> 


Vi'- 


,  tV 


-V 


PEDAGOGICAL  QUESTIONS  FOR  METACOGNITIVE  CONTROL 


Metacognition  is  a  higher-order  process  we  employ  to  guide  various  lower 
level  processes.  Bransford  (1979)  believes  that  metacognition  requires 
assessing  the  current  state  of  our  knowledge.  The  notion  he  alludes  to  is 
that  as  we  monitor  our  cognitive  activities,  we  somehow  evaluate  current 
states  through  checking,  verification,  and  testing  of  the  concepts  and  skills 
presented  to  us  in  our  daily  experiences  with  what  we  know  or  have  stored  in 
memory.  I  argue  that  student  question  asking  articulates  these  monitoring 
processes  during  skill  acquisition.  Brown  and  DeLoache  (1978)  discuss  the 
basic  skills  of  metacognition  used  to  control  learning  and  problem  solving 
as:  predicting  consequences  of  actions  and  events,  checking  results  of  our 
own  actions  (did  it  work),  monitoring  one's  ongoing  activity  (how  am  I  do¬ 
ing),  reality  testing  (does  this  make  sense).  After  all  but  one  of  the  basic 
skills  they  present,  a  self-directed  question  was  placed  in  parentheses  indi¬ 
cating  that  questioning  is  a  naturally  occuring  metacogni tive  process  we 
employ  to  monitor  our  cognitive  activities.  We  can  easily  imagine  a  ques¬ 
tion,  'Will  it  work?'  after  the  first  skill  Brown  and  DeLoache  list  to  moni¬ 
tor  predicting  consequences  of  actions  and  events. 

Feedback  is  essential  for  evaluating  our  knowledge  of  the  world  and 
ongoing  cognition.  Without  feedback  we  would  have  nothing  with  which  to 
compare  our  current  knowledge  state.  It  is  this  information  which  can 
serve  as  the  stimulus  for  our  asking  a  question  and  provide  a  state  condi¬ 
tion  we  can  use  to  match  against  what  we  have  stored  in  our  own  internal 
representations  in  memory.  In  computer-based  learning,  there  are  two  main 
feedback  sources,  instructional  and  environmental,  that  we  might  use  to 
evaluate  current  knowledge  states. 

Memory  retrieval  is  the  mechanism  used  to  access  what  we  know  as  we  com¬ 
pare  these  feedback  sources  with  what  is  being  learned.  This  process  in¬ 
volves  certain  steps  the  learner  undergoes  as  he  or  she  monitors  what  is 
being  learned:  1)  an  item  or  probe  is  presented  and  encoded,  2)  a  search 
process  is  then  begun  attempting  to  match  the  probe  with  associated  knowledge 
in  memory,  and  3)  a  decision  occurs  when  a  match  for  the  probe  is  made.  It 
is  when  a  match  cannot  be  made  that  provides  the  condition  for  question  ask¬ 
ing.  In  a  computer-based  learning  by  doing  environment,  probes  can  be  either 
the  current  task  subgoal  condition  or  system  feedback.  If  the  decision  re¬ 
sponse  is  'yes'  to  the  match  of  the  probe  with  the  desired  knowledge  stored 
in  memory,  an  action  can  be  initiated  by  the  student.  If  the  decision  is 
'no'  the  student  can  make  a  judgement  about  what  he  or  she  needs  to  have 
access  to,  and  ask  a  question,  assuming  the  learning  environment  included  a 
tutorial  dialogue  capability. 

Once  the  knowledge  structure  is  accessed  in  memory,  pedagogical  questions 
can  function  to  help  build  and  generate  a  more  complete  domain  representa¬ 
tion.  The  working  hypotheses  for  the  kinds  of  pedagogical  questions  students 
ask  as  they  monitor  learning  are  that:  1)  the  number  of  questions  asked  will 
change  in  relation  to  the  amount  of  practice  and  feedback  a  student  has  re¬ 
ceived;  and  2)  the  kind  of  question  asked  will  change  in  relation  to  the 
amount  of  practice  and  feedback  a  student  has  received.  The  first  working 
hypothesis  is  based  on  findings  that  support  the  notion  that  more  questioning 
occurs  in  the  early  stages  of  learning  (Chi  &  Glaser,  1980)  or  when  the  task 
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presented  to  the  student  is  novel.  The  last  working  hypothesis  is  based  on 
the  argument  put  forth  by  Miyake  and  Norman  (1979)  who  stated  that  the  abil¬ 
ity  to  ask  questions  is  based  on  the  level  of  completeness  of  a  student's 
internal  representation  for  the  knowledge.  Thus  as  a  student's  internal  rep¬ 
resentation  acquires  more  structure  and  completeness ,  the  type  of  questions 
asked  should  reflect  the  new  information  and  change  accordingly.  The  author 
is  preparing  a  report  that  presents  empirical  data  to  test  these  hypotheses 
(Swartz,  in  preparation^). 

The  memory  framework  in  which  pedagogical  questions  access  current  levels 
of  understanding  is  a  spreading  activation  mechanism.  Two  basic  functions 
for  the  questions  are  proposed:  1)  to  focus  activation  in  a  particular  loca¬ 
tion  of  the  knowledge  representation  and  maintain  this  information  in  working 
memory  during  processing,  and  2)  to  organize  and  structure  the  knowledge 
through  discrimination  and  generaliza tion  processes.  Activation  initiated  by 
question  asking  will  regulate  flow  of  activation  within  the  knowledge  struc¬ 
ture  that  is  accessed  during  processing.  Discrimination  and  generaliza tion 
processes,  articulated  through  question  asking  as  various  end  states  are 
compared  and  evaluated  with  what  is  being  learned,  will  serve  to  monitor 
acquisition  and  organization  of  the  knowledge.  Strengthening  processes  will 
be  reinforced  through  additional  questions  and  repeated  performance  in  simi¬ 
lar  states  as  practice  continues. 

In  discussing  the  framework  for  this  model,  it  is  necessary  to  make  some 
structural  assumptions  about  how  the  knowledge  is  represented  in  memory.  The 
particular  domain  considered  is  a  procedural  skill,  the  command  language 
Lotus  1-2-3.  While  the  human  representational  system  in  memory  is  far  from 
understood,  frameworks  for  both  declarative  and  procedural  knowledge  have 
been  proposed  (Collins  &  Loftus,  1975;  Anderson,  1983a).  I  assume  a 
hierarchical  goal-based  semantic  network  for  the  domain  in  order  to  facili¬ 
tate  discussion  of  the  question  asking  processes  hypothesized  as  a  function 
of  the  operating  pedagogical  question  system.  Thus,  the  proposed  conceptual 
network  for  the  domain  will  include  concept  nodes  for  both  declarative  and 
procedural  knowledge  (I  have  separated  these  two  knowledge  types  in  the  con¬ 
ceptual  representation  that  was  used  to  deliver  tutorial  instruction  and 
which  is  presented  later).  This  seems  plausible  when  we  consider  that 
learning  in  any  new  domain  begins  declare tively  (Anderson,  1983a)  with  lower 
order  components  of  the  knowledge  (Hayes-Roth,  1977)  that  develop  and 
strengthen  themselves  in  the  representation  with  practice  and  continued 
learning.  Thus  we  might  consider  the  concept  nodes  for  procedural  knowledge 
in  the  conceptual  representation  as  a  type  of  lower  order  declarative  compo¬ 
nent  before  the  actual  skill  is  acquired.  As  learning  progresses,  a  new 
structure  for  the  procedural  knowledge  will  be  created  and  stored  in  memory. 

A  network  representation  for  the  knowledge  similar  to  the  Collins  and 
Loftus  model  (1975)  is  assumed  which  groups  together  concepts  in  the  domain. 
The  concept  nodes  are  linked  to  subordinate  nodes  which  further  define  the 
superordina te  node  concept.  Links  carry  relational  meaning  between  the  con¬ 
cept  nodes  which  are  defined  by  the  nature  of  the  domain  (procedural,  goal, 
causal,  property/attribute  are  examples  for  procedural  command  language  do¬ 
mains  such  as  Lotus  1-2-3).  Other  link  and  node  types  may  be  created  during 
learning  as  the  student  gains  experience  and  more  detailed  knowledge  through 
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practice  and  question  asking.  The  procedural  concept  nodes  in  this  declara¬ 
tive  network  can  be  thought  of  as  pointers  to  another  knowledge  structure 
located  elsewhere  in  long-term  memory  (LTM)  that  will  develop  as  the  domain 
procedures  are  learned.  Understanding  how  these  question  asking  processes 
access  and  instantiate  the  knowledge  in  memory  will  permit  us  to  examine  more 
closely  how  the  knowledge  is  located,  transformed,  and  used  to  modify  the 
developing  domain  representation  during  skill  acquisition. 

Spreading  Activation  Framework  for  Question  Probes 

The  framework  discussed  here  follows  Anderson's  (1983b)  model  of  spread¬ 
ing  activation.  Sources  of  activation  can  come  from  a  variety  of  origins. 

In  a  computer-based  learning  by  doing  paradigm  the  most  obvious  source  of 
activation  is  the  task  goal  the  student  encounters.  This  is  very  similar  to 
Anderson  (1983a)  who  believes  that  current  task  goals  serve  as  constant 
sources  of  activation.  However,  if  the  source  is  not  focused  on,  the  activa¬ 
tion  effect  will  eventually  decrement.  On  the  other  hand,  if  some  other 
element  maintains  activation  and  provides  focus  on  a  particular  knowledge 
node,  then  activation  will  remain  constant.  Student-generated  questions  as 
probes  in  memory  serve  this  focusing  function.  They  identify  the  locus  of 
the  missing  knowledge  in  the  internal  representation  and  can  sustain  activa¬ 
tion  of  a  particular  knowledge  node  and  its  associated  elements.  This  focus¬ 
ing  function  will  serve  to  keep  the  activated  node  in  working  memory  (WM)  for 
processing  as  the  student  either  receives  and  processes  a  response  to  the 
question  or  continues  to  apply  problem  solving  strategies  to  the  current  task 
subgoal . 

Questions  to  Focus  and  Maintain  Activation 


Lachman,  Lachman,  and  Butterfield  (1979)  believe  that  when  a  person  needs 
a  particular  piece  of  information,  he  or  she  attempts  to  retrieve  it  directly 
from  memory.  I  will  argue  that  student  questions  can  serve  a  control  func¬ 
tion  to  activate  the  domain  network  in  the  area  accessed  in  a  failed  attempt 
at  retrieval  of  the  information  required  during  learning.  Questions  as 
probes  in  memory  can  control  the  spreading  activation  mechanism  that  is  trig¬ 
gered  automatically  by  the  constant  source  of  activation  emanating  from  the 
task  subgoal  and  shift  the  focus  to  other  areas  of  the  network. 


Any  question  can  serve  this  function  as  long  as  the  knowledge  asked  about 
pertains  to  the  instructional  domain.  Therefore,  'Is  this  right?'  is  a  veri¬ 
fication  question  that  does  not  refer  to  any  knowledge-specific  information 
in  the  developing  internal  representation  and  doesn't  function  to  focus  acti¬ 
vation  in  the  network.  Rather,  questions  of  this  sort  are  more  true  meta- 
cognitive  monitors  for  checking  results  of  one's  actions  (Brown  &  DeLoache, 
1978).  Similarly,  'How  can  this  be  right?'  might  be  most  appropriately 
interpreted  as  a  me tacogni tive  question  even  though  causal  or  explanatory 
responses  might  result  were  the  question  to  receive  a  response. 


Following  tutorial  instruction  where  all  of  the  information  in  the  con¬ 
ceptual  representation  has  been  presented,  a  student  then  engages  in  a  prac¬ 
tice  exercise.  To  illustrate  this  focusing  function,  we  can  consider  a 
student  who  is  trying  to  erase  some  data  from  the  computer  screen  in  a  Lotus 
1-2-3  exercise.  The  student,  in  response  to  the  task  subgoal  as  the  primary 
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source  of  activation,  is  unable  to  retrieve  the  appropriate  method  from  mem¬ 
ory  as  various  problem  solving  strategies  are  attempted  and  asks  a  question: 

How  can  I  erase  that  cell? 

By  identifying  the  knowledge  node  in  the  question,  we  assume  that  the  'eras¬ 
ing*  node  has  been  activated  in  the  student's  internal  representation.  Edit-  I 

ing  is  a  super  class  node  for  several  operators  and  methods  for  editing  or 
'erasing'  knowledge  in  Lotus  1-2-3.  In  the  domain  representation  the  edit  1 

node  is  a  superordinate  node  that  includes  various  operators  or  methods  for 
editing:  the  escape  and  backspace  keys,  and  the  re  type-reenter  method  (See 

Figure  1).  Because  associated  links  and  nodes  also  receive  activation 
(Collins  &  Loftus,  1975)  from  the  focused  node,  we  can  see  in  Figure  1  that 
all  three  editing  methods  are  partially  activated  by  the  question  initially. 

Any  new  information  about  the  correct  method  obtained  from  the  response  will 
shift  the  focus  to  the  referenced  node  and  the  new  information  can  be  proc- 
essed  and  stored  in  the  appropriate  area  of  the  activated  knowledge  struc¬ 
ture  . 

In  a  hypothetical  tutorial  dialogue  that  might  ensue,  the  student  could  ! 

pose  another  related  question  and  use  the  new  question  probe  to  shift  the  j 

focus  across  levels  of  the  knowledge  structure  and  maintain  activation  in  a 
particular  area  of  the  network  as  tutorial  responses  are  processed.  For  I 

example,  the  student  might  ask  about  other  methods  or  operators  for  editing:  i 

I 

Can  I  use  the  escape  key?  (No,  but  you  could  use  the  backspace  key.) 

Where  is  the  backspace  key?  (On  the  upper  right  corner  of  the  key 

board. ) 

In  this  way,  activation  moves  down  from  the  editing  node  to  its  lower-level 
nodes  making  these  other  items  available  in  WM  as  the  student  attempts  to  un¬ 
derstand  a  particular  problem.  This  process  of  expanding  questions  surround¬ 
ing  a  particular  node  can  be  thought  of  as  a  form  of  elaboration.  Reder  and 
her  colleagues  (1986)  report  that  elaborations  seem  most  important  to  the 
student  learning  to  select  and  execute  procedures  correctly.  Following  this 
notion,  we  can  see  that  student  questions  as  probes  in  the  network  may  also 
serve  an  elaborative  function,  however,  I  will  not  pursue  this  cognitive 
process  any  further  here. 

Another  example  of  the  focusing  function  is  in  the  case  where  retrieval  j 

processes  are  unable  to  transfer  the  needed  information  to  WM  from  LTM.  In  I 

this  case,  a  question  can  access  the  knowledge  structure,  maintain  activation  j 

at  a  particular  level,  and  serve  as  a  bridge  between  the  LTM  trace  and  WM.  ' 

Consider  a  student  who  encounters  a  task  goal  similar  to  one  accomplished  I 

previously  but  is  unable  to  retrieve  the  information  and  asks  a  question:  1 

i 

I  can't  remember  how  to  use  the  point-highlight  method.  | 

Through  this  question  the  activation  spreads  through  the  higher- level  knowl-  ! 

edge  node  where  this  method  should  be  stored,  Data  Entry,  and  descends  to  the 
subordinate  node  for  the  point-highlight  method  (See  Figure  2).  In  inter-  ' 

preting  the  question,  we  can  assume  the  student  believes  this  data  entry  < 
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Figure  1.  Semantic  network  model  used  for  tutorial  instruction 
with  a  Question  activating  the  network 
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Figure  2.  Question  activating  a  lcwer  level  in  the  network 


method  to  be  correct  and  so  articulates  the  need  for  assistance  in  carrying 
out  the  individual  steps  for  the  method.  We  can  infer  that  the  method  is  not 
yet  instantiated  in  the  domain  representation  at  this  stage  of  acquisition 
and  thus  cannot  be  transferred  to  WM  when  needed.  Here  the  question  accesses 
the  top  level  node  for  the  knowledge  sought,  descends  to  the  point-highlight 
node,  and  maintains  activation  so  that  the  appropriate  associated  information 
can  be  brought  into  WM  as  the  tutorial  response  is  given  enabling  the  stu¬ 
dent  to  carry  out  the  method.  With  the  appropriate  response  and  subsequent 
student  performance,  new  nodes  containing  the  information  necessary  to  carry 
out  this  method  will  be  added  and  strengthened  in  the  network. 

Novice  students  will  have  little  or  very  weak  associa tional  links  between 
nodes  in  their  knowledge  representations  initially.  Thus,  one  might  wonder 
why  focusing  activation  would  be  a  critical  element  for  facilitating  compre¬ 
hension  of  the  new  material.  Nevertheless,  there  are  certain  factors  that 
come  into  play  even  when  the  strength  of  association  in  the  network  is  not 
well  established.  When  multiple  input  sources  from  the  system  environment 
initiate  activation  in  the  developing  internal  representation,  students  may 
become  confused  and  cognitively  overloaded  by  these  various  stimuli.  Because 
the  domain  is  so  new  and  as  yet  not  well  organized,  students  may  be  unsure  of 
what  particular  feedback  source  over  and  beyond  that  activated  by  the  task 
subgoal  is  important.  Therefore,  questions  may  be  articulated  in  an  attempt 
to  focus  activation  on  the  perceived  relevant  knowledge. 

Consider  an  example  where  the  student  is  attempting  to  apply  a  multi-step 
method  for  entering  numbers  into  an  argument  range  as  part  of  a  calculation. 
The  immediate  goal  is  to  create  the  calculation,  however,  the  action  required 
to  accomplish  this  goal  generates  multiple  feedback  sources  from  the  computer 
display  which  can  be  confusing  to  the  novice.  Feedback  from  one  action 
prints  information  on  one  part  of  the  screen;  another  highlights  a  different 
region  as  the  range  is  specified.  These  system  responses  may  serve  to  in¬ 
crease  the  level  and  spread  of  overall  activation  during  processing  of  this 
subgoal.  A  student  may  choose  to  ask  a  question  to  focus  on  one  source  and 
consequently  control  the  activation.  For  example: 

Does  this  highlighted  area  indicate  the  cells  in  the  function? 

This  question  serves  to  focus  on  one  feedback  source  resulting  from  specify¬ 
ing  this  range  as  the  student  attempts  to  understand  how  this  data  entry 
method  works.  Through  such  question  asking,  a  student  is  able  to  control 
what  is  being  processed.  Now  that  I  have  described  how  questions  function  to 
focus  and  sustain  activation  in  a  particular  area  of  the  network,  we  can 
continue  with  the  second  question  function  of  discriminatory  and  generaliza¬ 
tion  processes  to  organize  and  structure  knowledge. 

Questions  as  Guides  for  Organizing  Knowledge 

Students  acquire  information  through  active  problem  solving,  dynami¬ 
cally  updating  the  representation  for  the  knowledge  through  repeated 
practice.  Reactive  environments  which  inform  students  of  their  success  or 
unsuccess  as  they  solve  problems  serve  to  foster  this  type  of  learning.  As 
students  reason  about  the  new  knowledge  being  learned,  they  may  engage  in 
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discrimination  or  generalization  learning.  Certain  types  of  pedagogical 
questions  articulated  at  specific  times  within  the  acquisition  process  can 
reflect  these  types  of  learning  as  students  evaluate  their  knowledge  state 
with  the  current  task  state.  As  the  knowledge  becomes  instantiated  in  the 
student's  internal  representation,  questions  will  function  to  organize  and 
strengthen  a  particular  aspect  of  knowledge  that  is  being  learned  (Swartz, 
1987;  Robertson  &  Swartz,  1987).  In  contrast  to  the  activation  focusing 
function,  only  certain  kinds  of  questions  function  to  organize  the  knowledge. 
Since  all  questions  accessing  the  domain  knowledge  function  to  focus 
activation  at  a  particular  location  in  the  network,  organizing  questions  will 
have  a  dual  functionality. 

Performance  feedback  that  is  not  understood  will  direct  the  questioner 
toward  a  certain  path  in  memory  dependent  on  the  active  problem  space  in 
which  he  or  she  is  interacting  by  providing  the  student  with  information  to 
evaluate  and  match  with  what  is  currently  in  the  knowledge  structure  (Collins, 
Brown,  &  Newman,  1987).  When  students  compare  this  feedback  with  their  own 
expectations  of  the  consequences  of  a  particular  action,  they  may  choose  to 
ask  a  question  to  confirm  these  expectations.  In  much  the  same  way,  the  in¬ 
structional  sequence  will  provide  the  student  with  situations  with  which  to 
evaluate  what  is  currently  stored  in  memory.  The  particular  time  within  an 
instructional  sequence  that  a  question  is  posed  will  define  search  control  in 
the  representation  based  on  the  completeness  of  the  knowledge  structure,  what 
is  known  about  the  goals,  operators  and  methods  or  the  level  accessed  in  the 
structure,  what  needs  yet  to  be  learned,  and  the  strength  of  association  for 
the  knowledge  or  how  well  it  is  learned.  As  learning  proceeds  through  the 
associa tional  stage,  pedagogical  questions  will  function  to  organize  the 
knowledge . 

As  novice  students  encounter  and  attempt  a  new  problem,  they  generally 
focus  on  localized  evaluations  of  immediate  consequences  of  their  actions 
(Anzai,  1987).  When  novices  compare  their  condition-action  states,  discrep¬ 
ancies  are  often  noted  because  of  student  error  in  applying  a  particular 
action  or  misconceptions  they  may  harbor.  Learning  by  doing  promotes  ex¬ 
ploratory  behavior  and  students  will  often  apply  problem  solving  strategies 
when  a  task  subgoal  cannot  be  met  because  the  skills  are  not  yet  acquired. 

At  such  times  when  their  problem  solving  strategies  do  not  help  them  exit  the 
'impasse'  (Brown  &  VanLehn,  1980)  the  student  may  choose  to  ask  a  question. 
This  identifies  the  student's  inability  to  find  a  correct  repair  solution  as 
the  motivational  basis  for  question  asking.  The  state  evaluation  process  in 
this  instance  can  be  used  to  discriminate  between  the  differences  observed 
and  those  expected  through  articulation  of  some  type  of  question  intended  to 
clarify  the  situation.  As  search  control  is  altered  by  feedback  and  contin¬ 
ued  learning  and  practice,  so  too  is  the  function  for  the  question.  Ques¬ 
tions  may  serve  to  discriminate  between  states  initially  and  later  on  to 
generalize  about  the  knowledge  as  students  attempt  to  understand  what  they 
are  learning. 


Discriminatory  Questions 

In  learning  research,  discrimination  learning  can  be  defined  as  comparing 
differences  between  positive  and  negative  exemplars  (Anderson,  1983a)  which 
are  subsequently  used  by  the  student  to  evaluate  and  properly  encode  the 


correct  conditions  for  applying  some  sequence  of  operations.  A  difference 
heuristic  is  applied  to  the  states  being  compared  that  isolates  discriminant 
features  between  them  (Carbonell,  1987),  and  this  information  is  subsequently 
used  to  reorganize  the  knowledge  stored  in  memory.  Consider  some  examples 
where  students  encounter  situations  were  the  consequence  of  their  problem¬ 
solving  strategies  (a  negative  instance  if  it's  errorful)  do  not  match  with 
the  desired  condition  (a  positive  instance).  As  the  student  evaluates  and 
attempts  to  understand  the  reason  for  the  discrepancy,  he  or  she  may  articu¬ 
late  a  question  to  aid  comprehension  of  the  detected  difference.  This  ques¬ 
tion  will  function  in  a  dual  role:  to  monitor  learning  via  discrimination 
and  to  solicit  instructional  feedback  to  explain  the  discrepancy. 

Questions  functioning  as  discriminators  during  skill  acquisition  will 
take  on  context-specificity  depending  on  what  type  of  information  the  student 
is  evaluating  and  the  nature  of  the  condition-action  consequences  resulting 
from  the  student's  problem-solving  behavior.  Thus  the  particular  state  a 
student  is  confronted  with  will  define  the  context.  For  example,  a  student 
learning  computer  applications  can  press  a  wrong  key  thus  resulting  in  an 
erroneous  state.  Attempting  to  understand  the  discrepancy  between  the  state 
anticipated  by  the  action  and  the  actual  state  that  resulted,  the  student  may 
pose  a  question: 

Why  is  that  symbol  printed  there? 

In  this  case,  instructional  information  is  solicited  and  once  the 
tutorial  response  is  received,  the  student  learns  that  an  error  has  been 
made.  (Why  questions  comparing  two  states  or  a  sequence  of  actions  are  in¬ 
dicative  of  seeking  causal  information).  With  this  explanatory  information, 
the  student  can  correct  the  mistake,  move  forward  in  the  task,  and  appropri¬ 
ately  encode  the  unit  of  knowledge.  Consider  another  situation  where  the 
student  is  learning  a  method  for  executing  a  copy  command.  The  method  re¬ 
quires  that  certain  operations  be  conducted  in  a  specific  order.  The  novice 
student  has  mistakenly  applied  these  operations  in  an  incorrect  order  result¬ 
ing  in  a  state  which  was  not  anticipated  by  the  student.  As  the  current 
state  (negative  instance)  is  compared  to  the  intended  state,  (positive  in¬ 
stance)  that  the  student  has  in  his  representation  for  the  knowledge,  the 
student  may  pose  a  question  to  evaluate  the  discrepancy. 

Shouldn' t  that  have  copied  the  cell? 

Again,  the  instructional  response  solicited  will  point  out  the  correct 
sequence  of  operations  required  by  the  method  the  student  wants  to  use.  In 
this  question,  a  verification  type,  an  implied  hypothesis  is  asserted  sug¬ 
gesting  that  the  student  expected  the  copy  behavior  to  have  occurred.  Previ¬ 
ous  work  in  pedagogical  question  asking  (Swartz,  1987)  has  noted  that  a  se¬ 
quence  of  casual-verification  type  questions  suggests  that  students  are  form¬ 
ing  and  testing  hypotheses  about  states  as  they  are  evaluated.  Thus,  as 
students  monitor  their  own  learning,  these  questions  can  facilitate  discrim¬ 
ination  learning  while  actively  acquiring  problem-solving  skills.  Another  way 
students  reason  about  the  knowledge  they  learn  is  through  an  inductive  process 
of  generalization.  Student  questions,  especially  the  casual-verification 
types,  can  also  be  used  to  articulate  generalizations  when  evaluating  states. 


Questions  that  function  to  generalize  knowledge  are  articulated  when 
commonalities  between  the  current  and  prior  conditions  are  located  and  evalu¬ 
ated.  In  generalization  learning,  the  positive  exemplars  encountered  are 
regarded  as  true  for  all  cases  until  a  situation  occurs  where  a  particular 
condition-action  state  doesn't  match  the  positive  instance.  Implicit  in  this 
type  of  learning  is  that  a  certain  amount  of  knowledge  is  currently  available 
to  enable  recognition  of  positive  examplars.  Consider  again  a  student  learn¬ 
ing  the  copy  command.  On  encountering  a  task  subgoal,  the  student  may  recog¬ 
nize  a  condition  where  he  or  she  thinks  the  copy  command  may  apply.  A 
question  to  verify  this  generalization  hypothesis  may  be  asked: 

Can  I  use  copy  on  this  column  if  I  want  it  (column  of  data)  to  go  over 

there? 

Providing  that  this  generalization  is  justified,  a  confirmatory  response 
will  validate  the  student's  hypothesis.  We  can  infer  from  this  type  of  ques¬ 
tion  that  the  student  believes  the  implicit  hypothesis  articulated  to  be  true 
(a  generalization  for  the  application  of  the  copy  command  from  a  different 
exercise),  but  is  not  completely  certain  and  seeks  confirmation  of  this  gen¬ 
eralization  through  a  verification  question.  With  a  confirmatory  response, 
the  student  can  move  forward  in  the  task  and  correctly  process  the  informa¬ 
tion.  If  on  the  other  hand  the  correct  response,  "No",  invalidates  the  hy¬ 
pothesis,  the  student  will  evaluate  the  negative  instance  and  stop  the 
generalization  with  the  knowledge  element  he  or  she  is  processing  through  the 
question.  Consider  another  situation  where  a  student  is  learning  a  method 
for  executing  the  same  command.  The  method  requires  that  certain  operations 
be  conducted  in  a  certain  order.  The  novice  student  has  mistakenly  applied 
the  operations  in  an  incorrect  manner  resulting  in  an  incorrect  and 
unanticipated  state.  As  the  current  state  (a  negative  instance  generated 
through  error)  is  compared  to  the  intended  state  (a  positive  instance)  the 
student  may  pose  a  question  to  guide  understanding: 

Shouldn't  that  have  copied  the  cell  like  before? 

Again,  the  instructional  response  the  student  is  soliciting  will  point  out 
the  correct  sequence  of  operations  required  by  the  method  needed  to  obtain 
the  task  goal.  The  negative  form  of  the  verification  question  implies  that 
the  student  believes  the  generalization  for  copying  to  be  true.  Indeed,  this 
is  a  valid  generalization  if  no  error  were  committed.  With  this  error 
pointed  out,  the  student  can  now  recognize  it  as  such  and  the  generalization 
for  the  knowledge  can  remain  intact. 

Following  Anzai's  (1987)  claim  that  novices  focus  on  local  consequences, 
it  seems  reasonable  to  expect  questions  to  function  for  discriminatory  pur¬ 
poses  prior  to  generalization  in  the  early  stages  of  skill  acquisition.  In 
order  to  generalize,  one  needs  to  have  a  more  established  representation  for 
the  knowledge  and  to  know  about  the  positive  exemplars  for  a  particular 
knowledge  element.  Extended  learning  and  practice  to  develop  and  organize 
the  knowledge  seems  a  logical  prerequisite  to  this  generalization  function 


for  pedagogical  questions.  Thus  the  use  of  generalizing  questions  will  be 
indicative  of  students  with  a  certain  amount  of  learning  and  practice  in  the 
domain. 

We  have  seen  through  these  examples  how  student-generated  questions  func¬ 
tion  to  focus  and  maintain  activation  as  the  knowledge  structure  is  accessed 
to  bring  information  into  WM.  The  knowledge  activated  in  the  network  is  then 
made  more  available  for  students  to  process  as  the  second  question  function 
to  organize  and  structure  knowledge  through  discrimination  and  generalization 
processes  occurs.  The  last  section  in  this  report  proposes  a  method  for 
deriving  knowledge  states  using  the  theoretical  framework  for  pedagogical 
question  asking  as  its  foundation.  With  the  description  of  the  pedagogical 
question  system  and  how  questions  function  to  access  knowledge  to  process  and 
organize  knowledge  in  memory,  we  are  now  ready  to  describe  a  model  for  knowl¬ 
edge  state  assessment  through  question  analysis  and  computation  of  the  acti¬ 
vation  patterns  generated  from  mapping  questions  unto  the  domain 
represents  tion . 

KNOWLEDGE  STATE  ASSESSMENT 

Evaluation  of  the  kinds  of  questions  asked  during  learning  should  provide 
information  that  is  descriptive  of  the  knowledge  state  for  a  student  as  the 
skill  becomes  acquired.  We  have  seen  how  students  evaluate  their  own  knowl¬ 
edge  states  prior  to  asking  a  question  in  a  monitoring  process  in  an  attempt 
to  understand  the  knowledge.  It  follows  that  we  might  use  the  sequence  of 
these  questions  and  the  activation  patterns  that  result  to  describe  what  the 
student  knows. 

In  the  theoretical  description  of  pedagogical  question  asking  I  have 
proposed,  the  spreading  activation  framework  in  memory  dictates  which  nodes 
in  the  knowledge  structure  will  be  activated  when  the  question  is  asked.  What 
is  sought  from  the  knowledge  structure  will  be  identified  by  the  particular 
knowledge  node  asked  about.  Specific  information  a  student  needs  about  that 
knowledge  will  be  reflected  in  the  type  of  question  utterance  that  is  used. 

The  question  interpretation  process  used  to  identify  question  types  is  made 
up  of  a  three  part  symbolic  representation  for  the  surface  structure  of  the 
utterance  that  defines  the  form  of  a  question  to  include  the  purpose,  con¬ 
text,  and  content.  Thus  by  keeping  track  of  the  questions  and  tabulating  the 
interactions  between  them  and  the  particular  knowledge  node  accessed,  we  can 
derive  a  model  of  the  student's  knowledge  at  any  time  during  instruction. 

Student  Model  Parameters 

The  model  of  a  student  derived  from  pedagogical  question  asking  must  take 
into  account  several  factors  if  it  is  to  be  a  reliable  measure  of  a  student's 
knowledge  state.  It  must  include  what  the  student  knows  and  does  not  know 
about  the  domain,  the  context  of  the  question,  its  purpose,  and  the  plan  to 
achieve  task  goals.  Modeling  a  student's  understanding  of  some  subject  mat¬ 
ter  is  conceptually  related  to  the  definition  of  the  domain  representation 
itself  (Burton  &  Brown,  1979).  The  question  interpretation  model  I  propose 
is  based  on  a  conceptual  definition  that  includes  slots  for  referencing  the 
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domain  knowledge  asked  about,  the  task  context,  and  one  which  describes  the 
predicate  relation  for  the  question  type.'  (See  Swartz,  in  prepara tiona ,  for 
more  details).  Knowledge  state  assessment  can  be  defined  by  considering  four 
parameters  for  the  questions  in  the  learning  task: 

1)  Purpose  of  the  question  (GOALS) 

2)  Location  of  the  question  in  the  instructional  problem  space  (CONTEXT) 

3)  What  knowledge  is  requested  (CONTENT) 

4)  Student  performance  used  to  achieve  task  goals  (PLANS) 

Sources  of  information  to  describe  a  student's  knowledge  state  in 
computer-based  learning  can  be  implicitly  inferred  from  student  problem  solv¬ 
ing  behavior  and  more  explicitly  inferred  from  the  questions  a  student  asks. 
The  formulation  of  student  knowledge  state  assessments  will  result  from 
analysis  of  the  tutorial  dialogue  that  occurs  in  an  instructional  sequence 
and  inferences  drawn  from  interpreted  questions  in  view  of  the  four  parame¬ 
ters  listed  above.  (Student  plans  inferred  from  question  asking  will  not  be 
addressed  here  in  detail,  however,  Robertson  and  Swartz  (1987)  examine  this 
topic  further.)  The  temporal  placement  of  a  question  when  matched  with  the 
student's  performance  behavior  and  their  sequence  in  a  task  exercise  can 
indicate  a  student's  plan.  They  can  also  indicate  when  he  or  she  has  reached 
an  impasse  in  the  attempted  plan  and  attempted  repairs  (Brown  &  VanLehn, 

1980)  when  subgoals  are  not  attained. 

The  kinds  of  information  necessary  for  a  comprehensive  student  model 
might  include  such  items  as  performance  levels  for  mastery,  number  of  errors, 
time  on  task  or  number  of  attempts  per  lesson,  to  name  a  few.  Information 
derived  from  the  questions  a  student  asks  alone  cannot  address  all  the  areas 
above,  but  in  combination  with  a  more  complete  modeling  system  for  student 
performance,  should  provide  for  a  more  rich  representation  of  the  student 
model.  Student  questions  in  conjunction  with  student  performance  will  pro¬ 
vide  the  tutor  with  student  feedback  that  is  necessary  for  more  accurate 
modeling  to  occur  and  subsequently  better  tutoring  and  instructional  deliv¬ 
ery. 


By  mapping  the  symbolic  representation  of  the  student's  questions  on  the 
knowledge  representation  and  calculating  the  subsequent  activation  patterns, 
we  can  derive  a  model  of  that  student's  current  knowledge  state,  and  from 
this  information,  one  may  be  able  to  infer  what  concepts  and  skills  are  at¬ 
tained  or  if  any  misconceptions  exist.  While  it  may  be  tempting  to  view  this 
knowledge  state  model  as  a  diagnostic  technique,  the  fact  that  the  usage  of 
this  linguistic  system  varies  widely  among  individuals  may  affect  its  general 
diagnostic  utility.  Nevertheless,  when  questions  are  asked,  we  want  to  be 
reasonably  certain  that  a  particular  knowledge  state  assessment  is  valid  as 
inferred  from  the  question  interpretation  process.  Certainly  in  combination 
with  other  diagnostic  techniques  in  tutoring  systems,  this  model  should  pro¬ 
vide  data  that  is  informative  of  what  a  student  does  and  does  not  know. 
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Question  Interpretation 


Question  decomposition  derived  from  this  interpretation  model  will 
result  from  a  surface  structure  analysis  of  the  question  utterance  and  pro¬ 
vide  a  symbolic  representation  for  the  functional  components  that  reflect  its 
semantic  purpose.  Each  question  phrase  is  analyzed  by  identifying  the 
interrogative  pronoun  and  placing  it  into  a  conceptual  category  to  reflect 
the  relation  function  of  the  question:  causal,  procedural,  property  and  so 
on.  These  'predicate'  relations  can  be  thought  of  as  defining  the  link  types 
used  to  associate  the  knowledge  in  the  domain  network  representation.  The 
propositional  unit  is  the  remainder  of  the  sentence  after  the  interrogative 
pronoun  is  removed.  This  propositional  unit  is  divided  into  two  functional 
components,  the  first  of  which  is  categorized  into  a  number  of  classification 
types  to  reflect  the  instructional  problem  space  context  encompassing  the 
proposition.  The  second  component  of  the  proposition  is  the  unknown  element 
of  the  question  or  what  Graesser  and  Black  (1984)  term  a  knowledge  element. 
The  knowledge  element  refers  to  the  instructional  content  the  questioner 
needs  to  have  to  complete  understanding.  The  general  model  proposed  for 
analyzing  pedagogical  questions  begins  with  a  decomposition  for  a  question 
into  the  following  three-place  form.  The  predicate  relation  connotes  the 
primary  question  function  and  takes  two  arguments:  context  and  content. 

(<Interrogative  PronounXProposi tional  Unit>)  — > 

(<Predicate  Rela tionXProposi tional  Context,  Knowledge  Content>) 
Question  Goals:  Classification  of  the  Question  Type 

The  purpose  of  the  question  can  be  inferred  from  what  kind  of  information 
is  requested.  While  the  goal  of  each  question  at  its  most  basic  level  is 
intended  to  obtain  instructional  information,  according  to  the  first  princi¬ 
ple  in  the  structure  of  the  general  question  system,  the  specific  type  of 
information  asked  for  more  explicitly  articulates  the  question  goal.  Al¬ 
though  the  best  expressed  goal  for  each  question  requires  consideration  of 
all  interpreted  components  of  the  question,  we  can  still  obtain  the  overall 
purpose  of  the  questions  using  the  predicate  relation  categories.  In  lin¬ 
guistics  and  artificial  intelligence  research  (Grice,  1975;  Pollack,  1985; 
Lehnert,  1978)  semantic  and  conceptual  categories  have  been  used  to  define 
question  predicates  (nominal,  causative,  agentive,  procedural,  etc.).  These 
predicate  types  can  serve  to  connote  the  function  or  purpose  of  a  particular 
question  for  obtaining  specific  kinds  of  information.  By  extension,  inter¬ 
preted  predicate  relation  types  identify  basic  question  goals.  A  student 
wanting  to  know  the  cause  of  some  action  or  consequence  will  ask  a  'Why' 
question  thus  inferring  a  causal  goal.  Similarly,  a  need  for  structural 
feature  information  implies  a  property  goal  for  'How  much/How  few'  questions. 


The  first  level  decomposition  defining  predicate  relation  categories 
takes  the  following  form: 

(CInterrogative  PronounXProposi  tional  Unit>)  — > 

(<Predicate  Rela  tionXProposi  tional  Unit>) 


I  adopted  a  conceptual  scheme  (Lehnert,  1978)  for  Interpreting  the  meaning  of 
these  interrogative  pronouns.  For  reasons'  of  parsimony  and  based  on  prelimi¬ 
nary  empirical  evidence  from  a  pedagogical  domain  (Swartz,  1987),  this  por¬ 
tion  of  the  model  has  been  modified  and  reduced  to  seven  conceptual  entities 
for  interpreting  the  question  types  used  in  procedural  skill  learning  and  an 
eighth  metacognitive  relation.  The  categories  and  their  matching  interroga¬ 
tive  pronouns  are  listed  below: 


Predicate  Relation 

Interrogative  Pronoun 

1. 

Goal 

Wha  t ,  Why 

2. 

Procedural 

How 

3. 

Causal 

Why 

4. 

Verification 

Can,  Do,  subject-verb  reversal 

5. 

Concept  Completion 

What,  When,  Who 

6. 

Proper ty/At tribute 

How  +  a  quantifier  (many,  few) 

7. 

Spatial  Orientation 

Where 

8. 

Me  tacogni tive 

(varies-used  for  rhetorical 

questions  where  no  response  is) 
needed) 

The  next  element  in  the  interpretation  model  is  the  analysis  of  the 
propositional  unit  which  is  classified  into  a  problem  space  context  sur¬ 
rounding  the  proposition  in  the  question. 

Question  Context;  Classification  of  the  Propositional  Unit 

When  a  question  is  articulated,  within  the  instructional  sequence  or 
problem  space  for  the  task,  will  identify  the  context  of  the  query.  In 
discourse,  propositions  express  the  main  idea  a  speaker  wishes  to  communi¬ 
cate.  However,  the  interpretation  of  questions  asked  in  a  situated, 
reactive  environment  such  as  computer-based  learning,  the  immediate  envi¬ 
ronment  and  its  features  more  so  than  discourse  features  will  be  used  to 
define  the  context  in  which  the  proposition  is  uttered.  Thus,  in  further 
decomposing  the  propositional  unit  expressed  in  the  question,  the  model 
takes  on  an  additional  part  to  describe  the  context: 

(<Predicate  RelationXPropostitional  Unit>)  — > 

(<Predicate  Relation>  <Problem  Space  Contexts) 

The  context  classification  process  depends  on  the  immediate  situational 
environment  the  student  is  in  when  the  question  occurs.  Because  the  learning 
paradigm  used  to  assess  this  interpretation  model  fosters  problem  solving 
strategies  as  the  student  learns  by  doing,  these  contexts  can  be  viewed  as 
problem  spaces.  The  identity  of  the  problem  space  context  in  conjunction 
with  the  problem  identification  process  a  student  undergoes  prior  to  uttering 
a  question,  principle  3  in  the  general  structure  of  the  question  system,  will 
help  interpret  the  overall  intent  of  the  question.  Goal  space  contexts  in¬ 
clude  questions  about  task  goals  and  subgoals.  Action  space  contexts  include 
questions  that  refer  to  operators,  operations  or  a  sequence  of  actions  re¬ 
quired  to  accomplish  a  specific  procedure  or  goal.  The  third  classification 
type  selected  is  a  state  space  context  since  problem  solvers  are  constantly 
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evaluating  states  as  they  search  for  the  correct  solution  path.  State  space 
contexts  refer  to  any  question  about  a  sta'te  transition  that  results  after 
some  operator  has  been  applied  to  some  existing  or  current  state,  a  particu¬ 
lar  goal  has  been  obtained  or  some  system  state  has  resulted.  Since  there 
may  be  a  merging  of  two  kinds  of  knowledge  in  this  task,  the  domain  knowledge 
of  the  command  language  itself,  and  the  device  knowledge  for  the  computer 
system,  learners  may  have  questions  that  refer  to  a  particular  state  transi¬ 
tion  that  could  be  part  of  either  system. 

A  set  of  simple  rules  is  provided  (Swartz,  in  preparation)  that  will  help 
classify  the  proposition  into  these  problem  space  contexts.  The  temporal 
placement  of  the  question  is  used  to  determine  whether  the  active  problem 
space  context  that  surrounds  the  question  is  a  goal,  action  or  state  space. 
Although  the  interpretation  model  relies  on  a  semantic  representation  for  the 
surface  structure  for  the  question,  syntax,  and  verb  tense  are  used  to  infer 
context  classification.  Once  the  problem  space  context  is  identified,  the 
last  element  in  the  model  can  be  classified. 

Question  Content;  Classification  of  the  Knowledge  Node 

The  third  element  in  the  interpretation  model  is  the  knowledge  node 
accessed  in  the  student's  internal  knowledge  representation  for  the  new 
skill.  This  knowledge  sought  through  the  student  questions  will  indicate 
which  part  of  the  knowledge  representation  is  accessed  or  the  content.  The 
last  element  in  the  interpreted  frame  is  thus  added: 


(<Predicate  Rela tion> <Propos ti tional  Unit))  --> 

(<Predica te  Relation)  <ProbIem  Space  Context,  Knowledge  Content)) 

As  the  representation  grows  and  acquires  more  levels  and  detail  in  its 
structure,  the  content  asked  about  should  access  different  levels  in  the 
network.  As  the  student  learns,  more  information  is  acquired  and  a  domain 
dependent  organization  for  the  knowledge  takes  form  and  develops  a  more 
elaborate  structure.  The  level  of  completeness  for  the  structure  will  effect 
the  level  of  search  when  a  question  is  formulated.  Several  examples  will 
illustrate  how  questions  can  probe  various  levels  of  the  knowledge  structure 
based  on  the  particular  knowledge  content  sought.  See  Figure  2  to  illustrate 
the  levels  accessed. 

How  do  I  use  the  copy  command? 

(<Procedural><Action  Space,  Copy  Command)) 

In  this  case,  the  entire  list  of  arguments  related  to  the  knowledge  node  for 
'copy  command'  is  unavailable  to  the  student.  Thus  all  steps  in  the  copy 
procedure  are  requested  in  this  question  implying  that  the  top  level  node  for 
copy  command  only  is  accessed. 

What  do  I  do  (in  'Copy')  after  I  point  to  the  first  element  (in  the 
range)? 

(<Concept  Comple tionXAc tion  Space,  Specifying  Parameters/Copy)) 
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In  this  case,  only  some  of  the  arguments  are  requested  through  the  ques¬ 
tion.  We  can  assume  by  the  question  that  the  student  knows  how  to  access 
the  menu,  select  the  command,  and  point  to  the  first  element,  but  fails  to 
retrieve  the  next  step  in  the  procedure.  Here,  the  information  missing 
can  be  thought  of  as  part  of  the  set  of  elements  or  leaves  descending  from 
the  knowledge  node  for  the  Copy  command. 

From  these  examples  we  can  briefly  see  that  different  levels  of  the  do¬ 
main  representation  in  memory  are  searched  based  on  what  the  question  asks 
about.  In  the  first  question  all  of  the  arguments  of  the  propositional  unit 
in  the  question  'use  the  copy  command'  are  missing.  The  knowledge  node  asked 
about  in  this  question  requires  that  all  of  the  leaves  in  the  set  of  elements 
that  make  up  the  proposition  for  expressing  the  procedure  for  using  the  copy 
command  are  requested.  The  second  question  illustrates  a  question  similar  to 
that  expressed  in  the  first  example,  however,  only  some  of  the  information  is 
requested.  We  might  infer  from  this  question  that  the  person  has  access  to 
the  preceeding  arguments  related  to  using  the  copy  command,  but  is  missing 
the  particular  step  required  for  anchoring  the  range. 

Question  Plans:  Location  in  the  Instructional  Sequence 

By  tracing  a  student  in  the  instructional  sequence,  we  can  view  the  stu¬ 
dent's  questions  in  relation  to  the  level  of  knowledge  obtained  through 
instruction  and  the  strength  of  association  for  the  knowledge  through  amount 
of  practice.  The  question,  along  with  the  performance  behavior  exhibited, 
will  indicate  what  solution  strategies  a  student  is  attempting  and  any  at¬ 
tempts  to  repair  (Brown  &  VanLehn,  1980)  them  during  skill  acquisition.  This 
performance  data  can  provide  information  about  the  student  plan  for  accom¬ 
plishing  task  goals.  Changes  in  the  kinds  of  questions  asked,  the  particular 
goals,  operators,  and  methods  a  student  is  applying  to  solve  the  task  prob¬ 
lem,  should  reflect  changes  in  his  or  her  plans. 

Novices  in  any  domain  would  not  be  expected  to  ask  questions  about  skills 
or  information  they  don' t  know  about.  Because  student  representations  may 
not  yet  appropriately  match  the  conceptual  domain  structure,  their  questions 
in  the  beginning  may  even  be  inaccurate.  This  notion  was  suggested  in  the 
possible  violation  of  principle  3  (problem  identification)  in  the  question 
system.  Thus  student  plans  inferred  from  question  asking  may  reflect  inap¬ 
propriate  or  incomplete  plans  in  early  phases  of  skill  acquisition. 

As  an  example  of  plan  inferencing  from  questions,  consider  the  implicit 
plan  of  action  from  the  questions  below.  After  a  30  minute  tutorial,  we  can 
assume  that  N  concepts,  procedures,  etc.,  have  been  presented  to  the  student. 
The  student's  performance  behavior  in  the  task  is  used  to  identify  the  cur¬ 
rent  subgoal,  creating  a  function.  This  will  be  the  first  item  in  the  stu¬ 
dent's  plan.  As  the  student  begins  the  plan,  a  question  is  asked: 

How  do  I  use  that  point- highlight  method? 

(<ProceduralXAction  Space,  Point- highlight>) 


At  this  point,  we  can  can  infer  a  second  step  in  the  plan:  using  the  identi¬ 
fied  method  (point-highlight)  for  obtaining  the  present  goal  of  creating  a 
function;  an  appropriate  method  in  this  case.  Furthermore,  although  the 
student  has  indicated  through  the  question  that  a  correct  method  has  been 
selected,  he  or  she  also  indicates  that  assistance  is  needed  in  order  to 
carry  out  the  plan  of  action.  Additional  questions  to  assist  in  accomplish¬ 
ing  the  current  subgoal  can  also  be  used  to  infer  steps  in  a  plan.  Assume 
the  student  had  not  mastered  all  the  steps  required  to  accomplish  the  method. 
During  execution  of  the  current  plan,  he  or  she  may  ask: 

Do  I  point  now? 

(<VerificationXAction  Space,  Pointer  Key>) 

At  this  point,  the  tutor  can  evaluate  the  plan  based  on  student  performance 
leading  up  to  the  question  and  provide  an  appropriate  response.  If,  given 
the  current  goal  and  plan,  this  question  were  inappropriate,  the  tutor  could 
respond  accordingly  and  help  the  student  revise  his/her  plan  plus  correct  the 
knowledge  that  the  student  is  assimilating  into  his/her  internal  representa¬ 
tion.  Weights  for  the  level  of  knowledge  and  amount  of  practice  (see  below) 
can  be  added  to  the  appropriate  subgoal  to  aid  in  tracing  a  plan. 

From  the  interpreted  questions  alone  we  are  able  to  identify  three  of  the 
four  parameters  necessary  for  modeling  the  student:  goals,  context,  and 
content.  Student  plans  inferred  from  question  asking  assumes  a  more  sophis¬ 
ticated  modeling  system  capable  of  tracing  student  performance  and  will  not 
be  addressed  any  further  here. 

Activation  Patterns  for  Knowledge  State  Assessment 

Activation  patterns  in  a  network  when  calculated  from  question  probes  at 
specified  intervals  will  reflect  which  nodes  are  being  currently  activated 
and  brought  into  WM  for  processing.  Level  of  knowledge  is  measured  by 
quantifying  the  number  and  type  of  instructional  task  goals  a  student  encoun¬ 
ters  in  an  instructional  session.  Strength  of  association  is  measured  by 
several  factors:  the  number  of  exercises  the  student  has  attempted,  the 
reoccurrance  of  similar  task  goals,  and  what  knowledge  is  being  questioned. 
Weights  can  be  added  to  the  appropriate  knowledge  nodes  to  compute  the  accre¬ 
tion  resulting  from  these  factors.  The  temporal  placement  of  the  question 
will  demarcate  where  the  student  is  within  the  instructional  sequence.  By 
matching  this  placement  to  the  student's  present  level  of  knowledge  and 
strength  of  association,  we  will  be  able  to  identify  how  much  knowledge  the 
student  has  in  his  or  her  internal  representa tion  and  what  organization  it 
has  as  compared  with  the  conceptual  representation  used  in  the  figures  in 
this  report. 

Use  of  Anderson's  (1983b)  spreading  activation  framework  will  allow  me  to 
discuss  the  thesis  that  questions  serve  to  organize  knowledge,  build  associ- 
ational  links,  and  strengthen  new  information  brought  into  a  developing 
knowledge  representation.  I  hope  to  provide  a  mechanism  by  which  we  can 
assess  how  knowledge  structures  come  to  be  constructed  during  learning  with 
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the  aid  of  question  asking.  I  do  not  want  to  make  claims  about  any  particu¬ 
lar  activation  theory,  and  use  this  framew'ork  only  as  a  vehicle  for  the  pres¬ 
entation  of  the  ideas  I  wish  to  express. 

Practice  and  Node  Strength  in  the  Network 

Spreading  activation  assumes  that  activation  moves  through  the  network 
providing  a  constant  but  varying  pattern  of  activity  reflecting  the  associa¬ 
tion  and  strength  of  the  links  and  nodes.  WM  can  contain  traces  from  LTM  and 
can  receive  links  to  traces  from  LTM  through  question- triggered  activation.  I 
have  presented  how  questions  can  function  to  focus  activation  in  a  network, 
thus  serving  as  a  bridge  to  the  LTM  trace  when  information  is  brought  from 
LTM  to  WM.  Anderson  believes  that  a  pattern  of  activation  in  LTM  is  set  up 
in  response  to  the  activation  accumulated  from  what  he  calls  focused  nodes.  I 
will  refer  to  these  focused  nodes  as  those  whose  activation  is  triggered 
through  question  asking.  The  level  of  a  node's  n^  total  activation  can  be 
derived  from  its  activation  a^y  to  a^y  received  from  nodes  n^  to  n^  yielding 
a  basic  summative  equation  Sa*  for  node  ny  Activation  that  a  node  ny  sends 
through  a  network  is  defined  by  the  strength  of  each  of  the  connected  nodes 
n^  to  n^  and  activation  level  for  the  node  initiating  activation.  Relative 
strength  for  the  node  must  be  determined  as  well  as  loss  of  activation 
through  decay.  The  level  of  activation  for  a  node  By  is  derived  using  the 
formula  below: 


See  Anderson  (1983b)  for  a  more  complete  discussion  of  this  model  and  the 
supporting  equations.  Here  qy  is  0  unless  it  is  a  focused  node  represented 
by  a  question  asked  in  which  case  a  value  indicating  the  amount  of  activation 
emanating  from  the  question  would  be  assigned  to  qy  and  added  to  the  equa¬ 
tion.  The  constant  source  of  activation,  ac  (where  a  task  subgoal  is  being 
processed),  will  have  a  value  if  present.  If  no  constant  source  is  present, 
ac  will  be  0  and  only  the  activation  from  the  associated  nodes  will  be  en¬ 
tered  into  the  equation  (2f  ax).  Thus  this  equation  should  reflect  the 
pattern  of  activation  resulting  from  the  questions  as  activation  focusers  in 
addition  to  the  constant  source,  ac ,  for  a  node.  Another  factor  central  to 
this  framework  is  that  the  strength  of  a  node  is  a  function  of  its  frequency 
of  exposure.  Following  Anderson's  model,  I  assume  that  the  strength  of  the 
focused  nodes  determines  the  amount  of  activation  emitted  into  the  network. 
Consequently,  the  stronger  the  node,  the  greater  the  activation.  Anderson 
offers  an  equation  for  summing  the  total  strength  of  a  node, 

S  =jti’b 

The  time  since  the  ith  strengthening  is  t^.  Practice  in  a  task  will  im¬ 
prove  performance  and  strengthen  the  associations  in  the  student's  inter¬ 
nal  representation,  thus  we  can  view  this  effect  through  the  following 
equation. 


where  s  is  assumed  to  be  spaced  repetitions  for  t  time  units  apart.  The 
strength  of  each  repetition  is  s  and  b ( <  1 )“  is  the  exponent  of  the  power 
function. 

Calculation  of  Activation  Patterns 

The  following  example  in  Figure  3  using  a  portion  of  the  conceptual  net¬ 
work  representation  for  the  domain  will  illustrate  the  activation  received 
from  a  question  in  relation  to  the  strength  for  the  accessed  node.  Consider 
a  student  faced  with  a  task  subgoal  of  entering  a  label,  BRIDGE,  into  the 
spreadsheet.  This  data  entry  subgoal  will  serve  as  the  constant  source  of 
activation  while  the  student  is  working  on  this  task.  I  assume  that  the 
student  has  selected  the  type-enter  method  (KE-14  node)  as  the  appropriate 
data  entry  procedure.  This  node  is  assigned  an  activation  level  of  1,  a,  for 
constant  source.  The  student  asks  a  question,  "How  many  spaces  are  in  a 
cell?",  and  the  activation  shifts  to  the  node  for  cell  address  KE-4,  that 
becomes  the  newly  focused  node.  It  too  receives  an  activation  level  of  1,  q ^ 
for  the  question  asked  in  the  activation  equation. 

The  strengths  for  each  node  are  denoted  in  the  figure  by  s^.  Since 
the  strength  of  node  KE-14  is  4  and  node  KE-4  is  2,  their  current  acti¬ 
vation  levels  will  take  4  and  2  units  of  activation  respectively.  Follow¬ 
ing  Anderson's  model,  activation  is  proportional  to  the  sum  of  the  node's 
strength.  This  is  calculated  using  the  strengths  from  the  associated  nodes. 
Therefore,  the  relative  strengths  for  KE-14  are  4/8  (KE-14/KE-4,2)  +  4/9 
(KE-14 /KE-14, 4)  +  4/9  (KE-14/KE-14 , 13 )  and  for  KE-4,  2/6  (KE-4/KE-4, 14)  + 

2/5  (KE-4/KE-4.2) .  The  activation  from  nodes  KE-4  and  KE-14  will  propogate 
through  the  network  and  spread  activation  to  the  associated  nodes  assigning 
relative  values  taking  into  account  the  distance  from  the  focused  nodes  and 
decay. 

To  illustrate  the  calculation  for  the  simple  activation  levels  for  the 
nodes  in  Figure  3,  I  will  first  make  some  basic  assumptions.  Because  ac¬ 
tivation  decays  when  the  node  is  not  focused,  I  will  select  an  arbitary 
value  of  .5  for  the  decay  factor,  d.  Calculating  activation  levels  that 
includes  the  reverberation  construct  becomes  difficult  even  in  a  small 
subset  of  the  domain  network  shown  here.  Therefore,  for  purposes  of  il¬ 
lustrating  this  model,  I  will  assume  a  direct  forward  flow  of  activation. 

The  equation  then  will  reflect  the  relative  strength  for  each  associated 
node  with  the  decay  factor  computed. 

The  total  activation  for  nodes  KE-4  and  KE-14  is  the  sum  of  their  own 
activation  (a  question  or  task  subgoal  as  a  source  of  activation)  plus  the 
summed  activation  received  from  the  associated  nodes: 

aKE-4  “2  +  2/6daKE-i4  +  2/5daKE_2 
aKE-14  “  4  +  4/8daKE-18  +  4/9daKE-4  +  4/9daKE-13 


v  r  vr  v  v  wy  vnr»rg  w 


In  this  way,  activation  levels  for  all  the  associated  nodes  in  this  portion 
of  the  network  can  be  derived.  If  we  add  'in  the  decay  factor  for  each  of  the 
associated  nodes,  we  get  the  following  values: 

aKE-4  “  2  +  ( .5)(  .33)aK£_1(i  +  ( .5)(  .4)aKE_2 

aKE-14  ”  4  +  ( . 5 ) (  . 5 )a^£_ +  ( .5)( .444)aEE_^  +  ( .5)(  .444)aEE_^.j 

These  levels  reflect  more  accurate  activation  when  accounting  for  the  natural 
decay  emanating  from  the  unfocused  nodes.  Simultaneous  equation  solutions 
give  the  values  of  5.85  and  10.194  for  the  two  nodes  respectively.  According 
to  Anderson,  information  from  node  KE-14  should  be  retreived  more  easily  than 
information  from  KE-4  based  on  its  higher  activation  level.  I  argue  that  in 
a  learning  situation  this  higher  value  permits  easier  access  to  the  informa¬ 
tion  in  LTM  as  it  is  brought  into  WM  for  local  processing.  By  extension, 
this  information,  since  it  is  easier  to  access,  would  become  instantiated 
first  in  a  given  representation. 


Summary 

Taking  a  series  of  interpreted  questions  from  some  learning  task  in  con¬ 
junction  with  the  task  subgoals  a  student  has  interacted  with,  we  can  calcu¬ 
late  the  current  knowledge  state  activation  pattern  by  applying  the  equation 


yi 


^axax  +  +  ac 


to  the  conceptual  network  representation  used  to  deliver  instruction.  By 
setting  a  threshold  criterion  for  a  strength  value,  s^  for  each  node  indi¬ 
cating  an  instantiated  state,  the  calculation  of  the  activation  patterns  will 
reveal  which  nodes  in  the  structure  are  instantiated  or  acquired  at  a  given 
time.  If  the  strength  value  for  a  particular  node  is  below  threshold,  its 
strength  will  be  the  sum  derived  from  the  total  sum  equation,  S  »2(it)-^. 


Each  time  a  particular  subgoal  representing  a  knowledge  node  is  presented 
to  a  student  within  an  instructional  sequence,  a  weight  can  be  assigned  to 
that  node  strengthening  its  place  in  the  knowledge  structure.  Simi¬ 
larly,  each  time  a  question  is  asked  a  weight  will  be  assigned  to  the  appro¬ 
priate  node.  These  weighted  values  can  be  used  by  the  student  modeler  to 
determine  knowledge  state  assessment  based  on  number  of  times  a  particular 
knowledge  node  has  been  presented  to  the  student  and  the  number  of  questions 
asked  per  node.  As  new  knowledge  nodes  are  presented  and  instantiated  in  the 
student  model,  the  knowledge  representation  derived  from  the  activation  pat¬ 
tern  will  take  on  more  levels  in  its  structural  makeup.  Work  is  underway  to 
assess  this  methodology  using  the  two  parameters,  ac  and  q^,  for  deriving 
knowledge  state  activation  patterns. 


CONCLUSION 


The  proposed  question  model  described  in  this  paper  is  intended  to  pro¬ 
vide  a  basis  for  further  investigation  into  the  cognitive  processes  that 
underlie  question  asking  and  the  usefulness  of  this  model  for  describing  a 
student's  knowledge  state.  Pedagogical  questions  function  as  a  control 
mechanism  to  focus  and  maintain  activation  of  knowledge  in  the  network  during 
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learning.  As  a  second  function,  these  questions  serve  to  make  explicit  the 
discriminatory  and  generalization  processes  a  student  is  engaging  in  as  in¬ 
formation  is  organized,  evaluated,  and  strengthened  in  the  student's  internal 
representation.  The  spreading  activation  framework  as  the  foundation  for 
understanding  how  these  questions  probe  memory  will  permit  researchers  to 
investigate  the  relevant  issues  for  how  and  where  the  knowledge  is  accessed 
and  represented  as  the  domain  structure  is  generated. 

I  have  argued  that  students  ask  questions  when  they  have  identified  a 
problem  (not  having  access  to  the  needed  information)  or  when  their  repair 
solutions  (Brown  &  Van  Lehn,  1980)  fail  during  problem  solving.  From  the 
model  described  here  and  these  two  arguments,  it  is  clear  that  students  must 
access  memory  prior  to  articulating  questions.  Through  the  computation  of 
activation  patterns  generated  from  question  asking  and  performance  behavior 
at  various  stages  of  skill  acquisition,  we  can  graphically  describe  a  stu¬ 
dent's  particular  knowledge  state.  By  identifying  what  level  of  knowledge  is 
accessed  and  how  these  activation  patterns  change  over  time,  we  will  be  able 
to  trace  the  development  of  a  student's  internal  representation  for  the  do¬ 
main  knowledge.  These  changes  will  thus  offer  evidence  in  support  of  the 
idea  that  knowledge  represen ta tions  change  qualitatively  as  learning  in¬ 
creases  (Hayes-Roth,  1977),  and,  the  identified  levels  from  a  particular 
knowledge  state  should  be  useful  for  predicting  the  kind  and  number  of  ques¬ 
tions  students  ask  (Miyake  &  Norman,  1979). 

Understanding  the  question  subsystem  and  how  it  operates  in  pedagogy  may 
also  have  important  implications  for  providing  tutorial  feedback  in  military 
training  systems.  We  have  seen  that  the  pedagogical  question  system  is  a 
specialized  communication  subsystem  which  operates  according  to  general  con¬ 
versational  rules  as  well  as  its  own  unique  set  of  principles  for  optimal 
functioning.  This  communication  subsystem  serves  the  purpose  of  exchanging 
instructional  information  between  student  and  tutor.  Analysis  of  the  ques¬ 
tions  military  students  ask  can  help  the  tutor  to  identify  what  particular 
concept  or  skill  a  student  is  processing  at  a  given  point  in  time.  With  this 
information,  precise,  more  adaptive  tutorial  responses  can  be  .provided  to 
individual  students.  This  instructional  feedback  will  serve  to  guide  mili¬ 
tary  students  in  better  comprehension  of  what  they  are  learning  and  enable 
the  tutor  to  correct  any  misconceptions  that  these  students  may  have  formu¬ 
lated.  Similarly,  this  feedback  should  help  military  students  in  their  prob¬ 
lem  solving  strategies  in  a  learning  by  doing  environment.  When  soldiers  are 
able  to  pose  their  own  questions  in  the  learning  task,  they  will  be  able  to 
control  instructional  delivery  according  to  their  individual  needs.  Finally, 
the  knowledge  state  information  derived  from  the  activation  pattern  calcula¬ 
tions  may  be  useful  for  student  modeling  in  computer-based  learning  to  iden¬ 
tify  what  a  student  knows  and  does  not  know.  However,  the  student  modeling 
problem  is  a  very  hard  one  to  solve  and  it  is  unclear  at  present  how  useful 
the  question  asking  data  alone  will  be  in  the  absence  of  more  sophisticated 
modeling  techniques. 

Current  psychological  research  is  beginning  to  examine  this  model  of 
question  asking  as  a  first  step  toward  understanding  the  cognitive  compo¬ 
nents  involved  in  student  question  asking.  This  work  (Swartz,  in  prepara¬ 
tion^)  will  attempt  to  provide  evidence  in  support  of  the  theory  of  peda¬ 
gogical  question  asking  so  that  we  may  understand  how  this  metacognitive 


29 


M  t.  It 


process  facilitates  knowledge  acquisition.  Future  work  is  planned  to 
asses  the  usefulness  of  calculating  activa-tion  patterns  in  order  to  define 
knowledge  states.  Only  when  we  can  understand  what  information  students 
request  and  how  their  questions  access  knowledge  in  memory  can  we  begin  to 
understand  how  student  question  asking  in  tutorial  dialogues  functions  to 
facilitate  learning.  This  research  problem  is  a  difficult  one;  the  pro¬ 
posed  model  is  a  first  attempt  toward  solving  it. 


VV.’*  . 
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